Supervisor-rated Job Performance Research Articles

An updated meta-analysis of the interrater reliability of supervisory performance ratings.

Given the centrality of the job performance construct to organizational researchers, it is critical to understand the reliability of the most common way it is operationalized in the literature. To this end, we conducted an updated meta-analysis on the interrater reliability of supervisory ratings of job performance (k = 132 independent samples) using a new meta-analytic procedure (i.e., the Morris estimator), which includes both within- and between-study variance in the calculation of study weights. An important benefit of this approach is that it prevents large-sample studies from dominating the results. In this investigation, we also examined different factors that may affect interrater reliability, including job complexity, managerial level, rating purpose, performance measure, and rater perspective. We found a higher interrater reliability estimate (r = .65) compared to previous meta-analyses on the topic, and our results converged with an important, but often neglected, finding from a previous meta-analysis by Conway and Huffcutt (1997), such that interrater reliability varies meaningfully by job type (r = .57 for managerial positions vs. r = .68 for nonmanagerial positions). Given this finding, we advise against the use of an overall grand mean of interrater reliability. Instead, we recommend using job-specific or local reliabilities for making corrections for attenuation. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Multidimensional work-nonwork balance: are balanced employees productive at work and satisfied with life?

Given ever increasing work and nonwork demands, achieving work-nonwork (WNW) balance is an important priority for many employees. Scholars have only recently settled on a definition of WNW balance as multidimensional and, as such, our understanding of its antecedents and outcomes is limited. Drawing on Conservation of Resources theory, we explore how organizations can support employees to achieve WNW balance and whether ‘balanced’ employees are more productive at work and satisfied with life. In detail, we hypothesize that the positive effect of supervisor WNW support (FSS) on employees’ life satisfaction and job performance is mediated by multidimensional WNW balance. We find, across two studies with two waves each, that only the dimension of WNW balance effectiveness and not the dimension of WNW balance satisfaction mediated the relationships between FSS, life satisfaction (Study 1 and 2) and self-rated job performance (Study 1). The relationship between FSS and supervisor-rated job performance (Study 2) was not mediated by either WNW balance dimension. As such, organizations can facilitate WNW balance through FSS, while ‘balanced’ employees seem indeed happier with their life and consider themselves to be better performing at work. We discuss the unexpected finding regarding the superior role of WNW balance effectiveness over WNW balance satisfaction for our outcomes in relation to the conceptualization of WNW balance as multidimensional and delineate important theoretical and practical implications.

Losing the meaning of being a socially responsible service worker: Moderating effects of customer and coworker incivility

This study investigates how a service company's corporate social responsibility (CSR) activities enhance frontline service employees' (FSEs) job performance and the contextual variables that moderate the effect of CSR on FSEs. It is proposed that FSEs' perceptions of their company's level of CSR involvement, called CSR perceptions, positively affect their customer orientation and job performance. It is also suggested that two types of harmful workplace contexts—customer and coworker incivility—reduce the positive impact of CSR perceptions. Three-month time-lagged data collected from 435 frontline employees in 21 hotels in South Korea showed that FSEs' CSR perceptions lead to higher levels of customer orientation and supervisor-rated job performance. It was also shown that customer incivility moderates the positive effect of CSR perceptions but coworker incivility does not. These results suggest that negative experiences related to customers decrease the positive effect of CSR more than those related to the workplace atmosphere or culture. Further, the theoretical and practical implications of the results are discussed.

Authentic leadership and employees’ job performance: mediation effect of positive employee health

In light of global competition and the COVID-19 pandemic, organizations are encountering an increasingly challenging and unpredictable environment. Consequently, employees are experiencing heightened levels of job strain. This study aims to explore the impact of various organizational mechanisms on promoting positive employee health within the organization, ultimately affecting employees’ job performance. The findings of this study indicate that authentic leadership and the absence of organizational politics are significant predictors of positive employee health. Moreover, positive employee health has a positive influence on supervisor-rated job performance through its effect on job engagement. This study serves as a valuable resource for organizations, shedding light on the fundamental factors that contribute to positive employee health. It also raises managers’ awareness of the importance of nurturing and sustaining employees’ emotional and physical well-being to maintain competitiveness in the market.

A matter of when, not whether: A meta-analysis of modesty bias in East Asian self-ratings of job performance.

Much attention has been paid to the question of whether there is a modesty bias in East Asian employees' self-ratings of job performance (i.e., a tendency to self-rate their performance lower than supervisors rate it). However, empirical results are conflicting, with some studies supporting the modesty bias and others not supporting it. We suggest that moderators representing boundary conditions for the modesty bias effect may shed light on these conflicting results. In essence, the question should not be "whether there is a modesty bias," but rather "when is there a modesty bias?" We propose three moderators: purpose of the ratings (administrative, developmental, or research), job performance dimension (task performance, organizational citizenship behavior, or leadership), and country-level in-group collectivism. Based on 40 studies (63 independent samples) with samples from East Asia (mainland China, Japan, South Korea, and Taiwan), we found no evidence of a modesty bias. That is, East Asian employees' self-ratings were, on average, higher than supervisor-ratings of job performance (i.e., a leniency bias). The one exception was when ratings were collected for research purposes; in this case, there was, on average, no mean difference between self- and supervisor-ratings. Thus, East Asian employees' research-purpose self-ratings are more modest, but this does not cross into a "modesty bias." In all, our results do not support a modesty bias as a widespread cultural norm among East Asian employees. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Too good to be true? Are supervisor-perspective ratings a valid substitute for actual supervisor ratings?

Due to well-known problems with self-ratings of job performance (e.g., inflation, weak correlation with supervisor ratings) and the challenges of collecting supervisor ratings of job performance, researchers sometimes use supervisor-perspective ratings (e.g., "how do you think your supervisor would rate your job performance?") instead. The assumption is supervisor-perspective ratings are less affected by the noted issues with self-ratings and therefore are more similar to actual supervisor ratings than traditional self-ratings. In fact, a considerable number of researchers have used supervisor-perspective ratings as an alternative to actual supervisor ratings. The purpose of this study is to meta-analytically determine the degree to which supervisor-perspective ratings are a valid substitute for actual supervisor ratings and identify the boundary conditions for this substitution. Our meta-analyses demonstrate that supervisor-perspective ratings are generally not a viable substitute for actual supervisor ratings. This is especially the case when (a) citizenship performance is measured, (b) data are collected in collectivistic cultures, and (c) all study data are gathered from the same source. We recommend not using supervisor-perspective ratings as a substitute for actual supervisor ratings. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

The Relationship between Work, Health and Job Performance for a Sustainable Working Life: A Case Study on Older Manual Employees in an Italian Steel Factory.

Supporting and retaining older workers has become a strategic management goal for companies, considering the ageing of the workforce and the prolongation of working lives. The relationship between health and work is especially crucial for older workers with manual tasks, considering the impact of long-standing health impairments in older age. Although different studies investigated the relationship between work ability and job performance, few studies have analysed the impact of workers' capability to balance between health and work demands, including managerial and organisational support (work-health balance). Considering health as a dynamic balance between work and health demands influenced by both individual and environmental factors, we assess the mediator role of work-health balance in the relation between work ability and job performance, both self-reported and assessed by the supervisor. The study utilises data from a case study of 156 manual workers, who were 50 years old or older and employed in a steel company in Italy. Data were collected inside the company as an organiational initiative to support age diversity. The findings show that work-health balance partially mediates the relationship between work ability and self-rated job performance, while it does not mediate the relationship with job performance as rated by the supervisor. Supervisor-rated job performance is positively associated with work ability, while it decreases with the increasing perceived incompatibility between work and health. A perceived balance between health and work is a strategic factor in increasing manual older workers' job performance. For older workers, not only the perceived capability to work is important but also the organisational health climate and supervisor's support. More studies are needed to verify if managers overlook the importance of health climate and support, as strategic elements that can foster performance for older employees.

Theory-driven game-based assessment of general cognitive ability: Design theory, measurement, prediction of performance, and test fairness.

Games, which can be defined as an externally structured, goal-directed type of play, are increasingly being used in high-stakes testing contexts to measure targeted constructs for use in the selection and promotion of employees. Despite this increasing popularity, little is known about how theory-driven game-based assessments (GBA), those designed to reflect a targeted construct, should be designed, or their potential for achieving their simultaneous goals of positive reactions and high-quality psychometric measurement. In the present research, we develop a theory of GBA design by integrating game design and development theory from human-computer interaction with psychometric theory. Next, we test measurement characteristics, prediction of performance, fairness, and reactions of a GBA designed according to this theory to measure latent general intelligence (g). Using an academic sample with GPA data (N = 633), we demonstrate convergence between latent GBA performance and g (β = .97). Adding an organizational sample with supervisory ratings of job performance (N = 49), we show GBA prediction of both GPA (r = .16) and supervisory ratings (r = .29). We also show incremental prediction of GPA using unit-weighted composites of the g test battery beyond that of the g-GBA battery but not the reverse. We also show the presence of similar adverse impact for both the traditional test battery and GBA but the absence of differential prediction of criteria. Reactions were more positive across all measures for the g-GBA compared to the traditional test battery. Overall, results support GBA design theory as a promising foundation from which to build high-quality theory-driven GBAs. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Pre-Employment Integrity Testing Using the Critical Hire-Screen and Step One Survey II

ABSTRACT Violence in the workplace, unethical conduct, illegal behavior, and employees with poor integrity are problematic for any industry. When it comes to working in community-based corrections (CBC), however, poor integrity can cause serious threats to the safety of the employee, their peers, their clients, and the community they serve. The Critical Hire-Screen (CH-S) and Step One Survey II (SOS II) are two pre-employment integrity assessments used in various CBC agencies. The present study explored the degree to which the CH-S and SOS II correlated with and predicted supervisor ratings of correctional officer job performance. Disparate impact analyses were also conducted exploring the degree to which each tool’s scale scores differed based on applicant age, gender and racial/ethnic status. Results revealed that only CH-S scale scores correlated with, and significantly differentiated between, supervisor ratings of job performance. Results also revealed that the CH-S and SOS II did not generate scale scores that significantly differed based on age, gender, or racial/ethnic status. Implications for correctional agencies are provided in this study.

Facebook Use and Job Performance: The Mediating Effects of Social Capital, Knowledge Sharing, and Job Satisfaction

ABSTRACT Social networking sites (SNSs) are extensively used worldwide, and the number of workers who use them in their workplaces is increasing. This study empirically examined whether social capital, knowledge sharing, and job satisfaction can mediate the effects of Facebook use on job performance. The results demonstrated that Facebook use positively affected both bonding and bridging social capital. In addition, both types of social capital have positive effects on online and offline knowledge sharing. While offline knowledge sharing and job satisfaction positively affected perceived job performance, online knowledge sharing negatively affected perceived job performance. On the other hand, offline and online knowledge sharing did not have a statistically significant effect on supervisor-rated job performance. Only job satisfaction had a positive effect. This study’s findings provide practical implications for financial companies that are considering policies related to the use of SNSs and want to enhance their employees’ job performance.

Testing the value of expert insight: Comparing local versus general expert judgment models

AbstractExpert judges often claim to utilize expert insight to tailor judgments to maximize predictive validity for a specific context. We evaluated multi‐organizational assessment data regarding the prediction of supervisory ratings of job performance from ratings on individual assessment dimensions, finding no evidence that the average expert assessor effectively tailored judgments to specific organizations to maximize prediction. Expert judgment was outperformed in all organizational contexts by linear models of expert judgment, optimal weighted regression models, as well as simple sum composites. Critically, the dimension weighting policies of the expert assessors were not consistent with optimal weights for predicting job performance at any organization. We discuss why expertise tends not to contribute to predictive validity and describe methods for improving overall judgmental accuracy.

A Personality-Based Measure of Employability

Abstract. Hogan et al. (2013) proposed a personality-based model of employability that describes individual differences in (1) being rewarding to deal with, (2) being able to learn the job, and (3) being willing to work hard. In this study, we evaluated the model by selecting subscales from the Hogan Personality Inventory (HPI; Hogan & Hogan, 2007 ) that best predicted supervisor ratings of competencies related to these three constructs. The psychometric properties of those scales were examined in independent samples. Results indicated that the scales converged with similar scales from other instruments, covaried in meaningful ways with observer descriptions, and predicted supervisor ratings of job performance. The measure – which is 64% shorter than the full HPI – includes personality characteristics applicable to most jobs across multiple job families that can be used to identify successful candidates.

Actions define a character: Assessment centers as behavior‐focused personality measures

AbstractTo expand our knowledge of personality assessment, this study connects research and theory related to two common selection methods: assessment centers (ACs) and personality inventories. We examine the validity of personality‐based AC ratings within a multi‐method framework. Drawing from the self‐other knowledge asymmetry model (Vazire, 2010), we propose that AC ratings are suited to capture personality traits that are observable in social interactions, whereas other methods (i.e., self‐ratings) are useful to assess more internal traits. We obtained data from two personality‐based ACs, self‐ and other‐rated personality inventories, and supervisor ratings of job performance. Confirmatory factor analyses indicated that personality‐based AC ratings reflected the Big Five traits. Consistent with the self‐other knowledge asymmetry model, AC ratings of more observable personality traits (Extraversion, Agreeableness, and Intellect/Openness) were correlated with inventory‐based measures of these traits. AC ratings demonstrated incremental validity in predicting job performance over inventory‐based personality measures for some traits (including Agreeableness, and Intellect/Openness) but self‐ratings also demonstrated incremental validity over AC ratings (for Conscientiousness). This implies that different personality measures capture unique information, thereby complementing each other. Yet, AC effect sizes were modest, suggesting that running personality‐based ACs is advisable only under specific circumstances.

Ambitious employees: Why and when ambition relates to performance and organizational commitment

It is often assumed that ambition has important workplace outcomes, but empirical research has only partially addressed this issue and frequently relied on imprecise measures of ambition. In two studies, based on an improved measure of ambition as a general disposition, we clarified how, why, and when ambition relates to performance evaluations and organizational commitment outcomes. Study 1 suggests that ambition has significant reputation effects in that self-rated ambition was positively related to ambition rated by supervisors and spouse/life partners, based on 100 employee-supervisor-spouse/life partners triads. Moreover, supervisor-rated ambition, but not self-rated ambition, was significantly positively related to higher supervisor-rated job performance and promotability, beyond employee-rated proactivity and generalized self-efficacy. Study 2 focused on organizational commitment outcomes with a three-month time-lagged study with 194 employees. We found that ambition was positively related to higher affective organizational commitment beyond achievement striving, especially when more organizational career opportunities were perceived. However, controlling for perceived organizational career opportunities and achievement striving, ambition was also positively related to increased organizational turnover intentions. Overall, the studies suggest that ambition among employees is generally positive and indirectly beneficial for individual job performance evaluations, but also poses some risks to organizational retention management.

Relative effects of human capital, social capital and psychological capital on hotel employees’ job performance

PurposeThis study examines the relative effects of human capital (HC), social capital (SC) and psychological capital (PC) on hotel employees’ job performance.Design/methodology/approachA sample of 417 employees from seven five-star hotels in China was recruited for the study. Both exploratory and confirmatory factor analyses were used to confirm the quality of measurement structures. Stepwise regression was used to examine the relative effects of the three capitals on hotel employees’ job performance.FindingsPC was found to be the strongest predictor of self-reported job performance (SJP) among the three capitals under investigation. Education and work experience in the HC domain affected SJP, whereas SC dimensions did not. However, only education and work experience in the HC domain were found to affect supervisor-rated job performance (RJP).Practical implicationsGiven the impact of PC on hotel employee’s SJP, human resource managers should attend to this capital in staff recruitment, retention and training and development.Originality/valueThis study provides a holistic comparative lens to examine the relative contribution of the three capitals on hotel employees’ job performance. This will help to further clarify the roles played by each of the capitals in hotel service work, thus advancing the development of the theories underlying each of the three capitals.

How despotic leadership jeopardizes employees' performance: the roles of quality of work life and work withdrawal

PurposeThis study contributes to the leadership literature by examining how and when despotic leadership jeopardizes employees' performance. Specifically, we examine whether employees' job performance could be harmed by despotic supervision through employees' work withdrawal behaviour. Moreover, we investigate whether the quality of work-life (QWL) helps in toning down the harmful effects of despotic supervision on work withdrawal.Design/methodology/approachWe used a multi-wave research design with data collected from 195 employees and their supervisors working in Pakistan's manufacturing sector. At time 1, we measured the independent variable (i.e. despotic leadership) and moderator (QWL), whereas, at time-2, the mediator (work withdrawal) was tapped by the same respondent with a time interval of three weeks between them. At time 3, the outcome (supervisor-rated job performance) was assessed directly by the supervisors.FindingsThe results support the mediating effect between despotic leadership and employees' performance through an enhanced level of work withdrawal behaviour. The effect of despotic leadership on job performance via work withdrawal behaviour was found to be weaker among employees with a higher level of QWL.Practical implicationsFor individuals, QWL serves as an enhancement of personal resources to deal with despotic leaders at the workplace; for organizations, our study results alert managers and leaders at the workplace to address employees' need for QWL as this positive resource may discourage work withdrawal behaviour and stimulate employees to perform well in their jobs despite facing despotic supervision.Originality/valueThis study contributes to the leadership literature by introducing work withdrawal as an underlying mechanism to explain the despotic leadership – job performance relationship. Further, we examined how the harmful effects of despotic leadership can be toned down through the moderating variable of QWL thus having practical implications for both employers and employees.

Social identity theory and leader–member exchange: individual, dyadic and situational factors affecting the relationship between leader–member exchange and job performance

Purpose This study aims to integrate social identity and leader–member exchange (LMX) theory to investigate the processes and boundary conditions around LMX–performance relationships. Through the application of two leader–follower subsamples, the authors test three main objectives. What is the effect of multi-dimensional dyad value-congruence on LMX and how does congruence on these dimensions differentially influence leader and follower perceptions of LMX? In a subsample of followers including supervisor-rated performance, the authors develop a model that examines how individual values moderate the effect of dyad contact on supervisor-rated job performance mediated by follower LMX. Design/methodology/approach The participants for this study include graduate and undergraduate social work students who were taking part in a one-year work placement within a social work organization as well as their immediate supervisors. Across a four-month period, participants filled out measures of their supervisor contact, work values and LMX. Supervisor-rated performance was also included. Findings Findings from the dyadic subsample show that growth value congruence is a predictor of follower-rated LMX, with value congruence across all values having no effect on leader-rated LMX. Within a subsample of followers, findings suggest that follower-rated LMX mediates the relationship between dyad contact and supervisor-rated job performance, with individual work values moderating this effect. Originality/value The current study offers several contributions to the literature on LMX and job performance. First, in this study’s dyadic leader–follower sample, the authors extend propositions made by social identity theory around value congruence and LMX by offering support for a multi-dimensional and multi-target approach to questions of values and LMX. Second, within this study’s larger non-dyadic sample, the authors offer insights into previous conflicting findings around dyad contact and LMX, by offering support for the indirect effect of dyad contact on supervisor-rated performance via LMX. Third, within this second sample, the authors also extend the literature on values and LMX to show that the process through which LMX influences job performance is dependent on follower values.

Predicting Correctional Officer Job Performance: Criterion Validity for the Critical Hire-Screen

ABSTRACT The Critical Hire-Screen (CH-S) is a pre-employment, overt integrity test developed for law enforcement and correctional officer applicants. This article examined the CH-S’s criterion validity by exploring the degree to which the CH-S predicted job performance ratings. Results revealed that CH-S scores significantly correlated with and adequately predicted supervisor ratings of job performance with a sample of correctional officers. Multiple, empirically derived, cut scores are presented in this paper to provide practical utility of these findings to hiring agencies.

Within-Individual Age-Related Trends, Cycles, and Event-Driven Changes in Job Performance: a Career-Span Perspective

Past research on age-related differences in job performance have focused primarily between-person comparisons. In the present study, we examine within-individual changes in supervisor-rated job performance to examine the influence of age-related trends, cycles, and event-driven factors. Our analysis is based on an eight-wave dataset from a multiple-cohort sample of employees (N = 750) varying in age from 25 to 65 years. We used an age-sequential design to disentangle maturation effects from historical effects. Results showed that population-level, within-individual change in a general measure of job performance was characterized by an increase in the first phase of the career (workers of 25–30 years), and then by a progressive decline. Within-individual levels of job performance were generally higher for younger workers than for older workers, and mostly reflected the influence of population-level trends but some even-driven effects as well. Results were in line with predictions from Baltes and Baltes’s (1990) meta-theory of selective optimization with compensation and the effects of age-related losses on performance. Results also provide insights into understanding the job performance trajectory over the career span.

Having a Calling on Board: Effects of Calling on Job Satisfaction and Job Performance Among South Korean Newcomers

Despite increasing research on calling, how calling functions for those experiencing transition from school to work and how their calling prior to working relates to later well-being and job outcomes has been understudied. The current study explored effects of perceiving a calling on job satisfaction and job performance, as measured at organizational entry and 2 years after organizational entry. Using a time-lagged collection of a sample of South Korean newcomers, the results based on structural equation modeling revealed that perceiving a calling was positively related to supervisor-rated job performance. Job involvement, which was measured 1 year later, fully mediated the relation between perceiving a calling and job satisfaction, but the hypothesized mediating role of job involvement on the link between perceiving a calling and job performance was not supported. We also examined moderating roles of perceived organizational support and perceived person-job fit on the relation between perceiving a calling on job involvement and found that perceived organizational support facilitated the effects of perceiving a calling on job involvement. Implications of these findings are discussed.